{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "3b5c1f5d-1c27-4e2c-806a-d14054f0e3af",
   "metadata": {},
   "source": [
    "## ollama部署"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6359f2cf-b650-4936-ae95-e153dcafc51e",
   "metadata": {},
   "source": [
    "|        |          | LLM                        | Embedding                        | Reranker                        |\n",
    "| ------ | -------- | -------------------------- | -------------------------------- | ------------------------------- |\n",
    "| Ollama | base_url | http://localhost:11434/v1/ | http://localhost:11434           | http://localhost:11434          |\n",
    "|        | api_key  | NA                         | NA                               | NA                              |\n",
    "|        | model    | qwen3:8B                   | dengcao/Qwen3-Embedding-0.6B:F16 | dengcao/Qwen3-Reranker-0.6B:F16 |\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ea652e06-d652-4655-8376-3ba2b0e8e7f9",
   "metadata": {},
   "source": [
    "## vllm部署"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "74fef4dd-a103-42bf-8dd2-a408844d1b49",
   "metadata": {},
   "source": [
    "\n",
    "|      |          | LLM                      | Embedding                | Reranker                 |\n",
    "| ---- | -------- | ------------------------ | ------------------------ | ------------------------ |\n",
    "| vLLM | base_url | http://localhost:9992/v1 | http://localhost:8000/v1 | http://localhost:8001/v1 |\n",
    "|      | api_key  | token-abc123             | NA                       | NA                       |\n",
    "|      | model    | my_qwen3_14b             | Qwen3-Embedding-0.6B     | Qwen3-Reranker-0.6B      |"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "36cecbd4-4604-4d52-8154-85571b68e756",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "26d63752-6aca-496f-a5be-4e5cc97ae570",
   "metadata": {},
   "source": [
    "## Xinference部署"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c571f7f5-87bf-487e-b607-105f1dd896f3",
   "metadata": {},
   "source": [
    "\n",
    "|            |          | LLM                      | Embedding             | Reranker              |\n",
    "| ---------- | -------- | ------------------------ | --------------------- | --------------------- |\n",
    "| xinference | base_url | http://localhost:9997/v1 | http://localhost:9997 | http://localhost:9997 |\n",
    "|            | api_key  | NA                       | NA                    | NA                    |\n",
    "|            | model    | my_qwen3_14b             | my_qwen3_embed_0.6b    | my_qwen3_reranker_0.6b |"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "fa08630c-5889-4bf4-8f6a-f058a21769cc",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_community.llms import Xinference\n",
    "\n",
    "my_llm = Xinference(\n",
    "    server_url=\"http://120.79.252.32:9997\",\n",
    "    model_uid = \"my_qwen3_14b\" # replace model_uid with the model UID return from launching the model\n",
    ")\n",
    "\n",
    "llm.invoke(\"你是谁？\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "200fc3f4-0e15-48dd-8605-586b6a604852",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "文本嵌入维度: 1024\n",
      "嵌入向量数量: 4\n",
      "每个向量的维度: 1024\n",
      "\n",
      "每个向量的前十位：\n",
      "文本 '机器学习' 的前十维度: [0.003894022200256586, 0.002543519251048565, -0.005048206076025963, -0.1012057438492775, 0.028114624321460724, 0.09616617113351822, -0.051155153661966324, 0.04542522504925728, -0.03727905824780464, 0.014972030185163021]\n",
      "文本 '深度学习' 的前十维度: [0.00782281905412674, 0.006280106492340565, -0.005760041996836662, -0.08216144889593124, -0.0008188830106519163, 0.09460803121328354, -0.04723409190773964, 0.048143111169338226, -0.06824644654989243, 0.034630175679922104]\n",
      "文本 '自然语言处理' 的前十维度: [-0.003955102991312742, -0.06085530295968056, -0.0032644968014210463, -0.148334801197052, -0.03478851169347763, 0.03278841823339462, -0.04042812064290047, 0.04846128076314926, -0.06426529586315155, 0.029821066185832024]\n",
      "文本 '计算机视觉' 的前十维度: [0.028781482949852943, -0.009483064524829388, -0.004396460950374603, -0.08308970928192139, 0.003152501070871949, 0.08561170846223831, -0.029889119789004326, 0.06318634748458862, -0.05848315730690956, 0.039431825280189514]\n"
     ]
    }
   ],
   "source": [
    "from langchain_community.embeddings import XinferenceEmbeddings\n",
    "\n",
    "# 使用API代理服务提高访问稳定性\n",
    "my_embed = XinferenceEmbeddings(\n",
    "    server_url=\"http://120.79.252.32:9997\", \n",
    "    model_uid=\"my_qwen3_embed_0.6b\"\n",
    ")\n",
    "\n",
    "# 生成单个文本的嵌入向量\n",
    "text = \"这是一个测试文本\"\n",
    "query_embedding = my_embed.embed_query(text)\n",
    "print(f\"文本嵌入维度: {len(query_embedding)}\")\n",
    "    \n",
    "# 生成多个文本的嵌入向量\n",
    "texts = [\"机器学习\", \"深度学习\", \"自然语言处理\", \"计算机视觉\"]\n",
    "document_embeddings = my_embed.embed_documents(texts)\n",
    "\n",
    "print(f\"嵌入向量数量: {len(document_embeddings)}\")\n",
    "print(f\"每个向量的维度: {len(document_embeddings[0])}\")\n",
    "\n",
    "# 简化的打印方法：直接打印每个向量的前十位\n",
    "print(\"\\n每个向量的前十位：\")\n",
    "for i, embedding in enumerate(document_embeddings):\n",
    "    print(f\"文本 '{texts[i]}' 的前十维度: {embedding[:10]}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5fd33858-62b7-4e62-bd88-5cac6c7f1974",
   "metadata": {},
   "source": [
    "## 硅基流动(免部署)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dea18bf3-14a2-4619-a9fe-ac50a2ee925f",
   "metadata": {},
   "source": [
    "|             |          | LLM                                                         | Embedding                                                   | Reranker                             |\n",
    "| ----------- | -------- | ----------------------------------------------------------- | ----------------------------------------------------------- | ------------------------------------ |\n",
    "| siliconflow | base_url | https://api.siliconflow.cn/v1/chat/completions              | https://api.siliconflow.cn/v1/embeddings                    | https://api.siliconflow.cn/v1/rerank |\n",
    "|             | api_key  | Bearer sk-oyynmtyjrsguxrwqdrgyeepzackpwgdrndnzdlydxtjbswup- | Bearer sk-oyynmtyjrsguxrwqdrgyeepzackpwgdrndnzdlydxtjbswup- | Bearer sk-oyynmtyjrsguxrwqdrgyeepzackpwgdrndnzdlydxtjbswup-                                    |\n",
    "|             | model    | Qwen/Qwen3-8B                                               | BAAI/bge-m3                                                 | BAAI/bge-reranker-v2-m3              |"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a9621566-7cea-4254-b52e-830890d9bcf9",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "95b20d0b-683f-4f16-9139-d55e5651395c",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python (learn-llm-rag-easily)",
   "language": "python",
   "name": "learn-llm-rag-easily"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
